Augmented Reality Meets Fine Art

“Our goal is to help visitors connect to the themes of the exhibition in playful, visceral way, while also encouraging them to see their everyday surroundings in a new light.” Says Neal Benezra, the Helen and Charles Schwab Director of SFMOMA. “On a deeper level, the Interpretive Gallery also serves as a reminder that artists have always been interested in breaking down the boundaries between natural and artificial realities—anticipating much of the work in VR and AR being done today.”

Inspired by the visual puns and paradoxes found in Magritte’s works from this period, the Magritte Interpretive Gallery presents a series of altered and augmented “windows.” Each window invites visitors to be “seen,” while simultaneously obstructing and alerting their perception of reality.

Some windows function as digital mirrors in which the visitors’ reflections do not behave as expected. Other windows sense visitors’ presence and open a gateway into another, augmented reality. The windows become both portals and problems, challenging expectations of what could and should be seen.

With the use of advanced, depth-sensing cameras and motion-tracking technology, the digital scenes in the windows draw on Magritte’s visual strategies. They also add a temporal twist, engaging the visitor in a brief journey by suspending and altering reality and perception. The combination of visual design and human-computer interaction creates a seamless experience activated by the presence of the visitor.

“We wanted to make the technology disappear as much as possible,” said Charles Yust, Principal Design Technologist and frog’s Project Lead. “There’s a mountain of coding, hardware integration, and software integration but all of that is in the service of creating a sense of ‘magic’. That’s what Magritte was achieving with just oil and canvas and we wanted to honor that while taking a modern day approach.”
In the making of the Interpretive Gallery, frog worked with SFMOMA’s Curatorial, Digital, Marketing, Exhibitions, and Visitor Experience staff as well as content strategists, fabricators, and designers. Together, the team found a modern interpretation of “the real with the mystery that is in the real” that we think Magritte himself would have enjoyed.

“frog typically works in service of solving problems, answering questions or otherwise providing clarity for our clients,” said Oonie Chase, Executive Creative Director. “Working with SFMOMA, we had the rare chance to design in service of raising questions, and to do so with Magritte, the undisputed master of visual enigmas. Working with SFMOMA is a dream project for frog; the way the museum is using technology to spur and expand engagement with art is incredibly inspiring.”